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BRIEF  OUTLINE  OF  RESEARCH  FINDINGS 


During  the  period  supported  by  ARO  Grant  and  Contract  Numbers 
DAHC04-75-G0186,  DAAG  29-78-G-0097 ,  and  DAAG  29-81-K-0133,  we  have 
obtained  many  results  in  the  area  of  Nonlinear  Regression  with  emphasis  on 
Spline  Methods.  Our  results  have  been  published  (or  will  be  published)  as 
listed  on  pages  1-4.  In  order  to  avoid  redundancy,  this  list  will  also  be 
used  as  references  to  this  outline  of  reserach  findings.  Many  of  the 
results  that  have  been  reported  in  our  Semi  Annual  Progress  Reports  will 
not  be  repeated  here.  Only  some  of  the  most  important  results  are 
outlined.  These  results  can  be  roughly  divided  into  five  areas. 

1.  Splines  in  nonlinear  regression  and  approximation 

Our  first  result  in  this  direction  is  the  study  of  how  smooth  are 
best  (least-squares)  splines  appriximants  with  a  fixed  number  of 
(variable)  knots  when  the  data  functions  they  approximate  belong  to  some 
smoothness  class.  These  results  appeared  in  1977  [1].  We  proved,  for 
example,  that  if  the  data  function  is  continuous,  then  all  best  spline 
appriximants  (with  n  knots,  say)  are  also  continuous  and,  however,  there 
exists  an  infinitely  differentiable  function  which  has  no  twice 
continuously  differentiable  best  (or  least-squares)  spline 
approximants  with  n  (variable)  knots.  The  (first)  continuously 
differentiable  case  is  not  yet  settled. 


We  next  considered  the  uniqueness  problem  of  best  L  approximation 
from  second  order  spline  manifolds  with  n  (variable)  knots.  The  main 
results  appeared  in  1977  [4]  and  1978  [4].  We  proved,  for  example,  that 
if  the  data  function  f  is  strictly  convex  with  concave  log  f",  then 
there  is  uniqueness,  and  that  there  is  no  uniqueness  if  the  concavity 
condition  on  log  f”  is  omitted.  We  also  introduced  the  idea  of  eventual 
uniqueness  and  proved  that  “eventually"  (i.e.  for  sufficiently  many  knots) 
we  do  not  require  the  concavity  condition  to  guarantee  uniqueness.  Our 
results  here  have  been  generalized  by  our  Ph.D.  student  Jeff  Chow  in  his 
Ph.D.  thesis  (S[5]). 

One  advantage  that  families  of  spline  functions  possess  over  other 
approximating  manifolds  arises  from  the  nonlinear  parameters  (the  knots) 
which  can  be  concentrated  in  regions  of  somewhat  complex  structure  of  a 
function  to  be  approximated.  This  had  been  recognized  by  de  Boor  and  his 
student  Dotson  who  developed  heuristics  to  obtain  an  asymptotically 
optimal  solution  to  an  ordinary  differential  equation  using  collocation 
and  knot  redistribution.  Our  papers  1978  [5]  and  1982  [3]  represented  the 
culmination  of  our  research  into  the  asymptotic  behavior  of  best 
approximating  splines  with  free  knots.  These  results  put  the  de  Boor- 
Dotson  work  on  a  firm  foundation.  Perhaps  surprisingly,  the 
asymptotically  optimal  knot  distribution  is  the  same  for  splines  as  it  is 
for  (discontinuous)  piecewise  polynomials.  Of  course,  the  approximation 
rate  is  the  same  but  the  constants  are  different.  The  major  result  may  be 
stated  as  follows.  Let  E(n,k,p,f)  be  the  error  (1  <  p  <  •)  of 

best  aproximatlon  to  a  function  f  e  C  [0,1]  (this  smoothness  assumption 


can  be  weakened)  from  the  C  splines  of  order  k  with  n  variable 

V(fc) 

n,k,p,f)  J(k,p,f  ).  Furthermore, 

(k) 

we  can  exhibit  an  Increasing  function  t  depending  on  f  so  that  if 

k-2 

we  approximate  by  splines  of  order  k  (in  C  )  with  knots  at 
{ t(i/N)}^Q,  the  error  in  best  approximation  E(n,k,p,f,t)  satisfies 

nkE(n,k,p,f ,t)  ♦  J(k,p,f ^). 

Another  area  on  nonlinear  regrssion  with  emphasis  on  spline  manifolds 
is  shape-preservation  approximation  with  optimal  (i.e.  Jackson)  degree  of 
approximation.  Our  results  in  this  direction  appeared  in  1980  [2]  and 
1980  [4].  We  extended  DeVore's  result  in  S [ 1 1 ]  to  all  spaces, 

1  _<  p  «°,  and  gave  an  elementary  and  constructive  proof.  Our  method  of 
proof  is  now  considered  very  important  by  others  and  has  been  used  by  many 
authors,  including  R.  Beatson,  D.  Leviatan,  H.  N.  Mhaska,  in  obtaining 
various  constraint  approximation  results. 

2.  Best  local  approximation,  Pade  approxlmants ,  and  digital  filters 

In  studying  splines  or  peicewise  polynomials  with  many  knots,  (or 

break-points)  one  is  quickly  lead  to  the  problem  of  best  approximation  on 

ever  decreasing  intervals.  We  call  this  best  local  approximation.  When 

rational  functions  are  considered,  J.  L.  Walsh  (S[14],  S[ 1 5 J )  already 

related  this  problem  to  Pade  approxlmants.  In  S [ 10 J  (see  also  1976  (2] 

in  S[l]),  we  introduced  the  notion  of  best  local  approximation  and  in  1978 

2 

[1]  we  discovered  various  results  in  the  L  setting.  A  detailed  study 


was  given  by  our  student  L.  Y.  Su  in  S [ 1 3 ] .  For  two  and  more  points,  the 


best  local  approximation  results  for  L  were  discovered  in  1981  [1].  We 
noted  in  particular  that  this  leads  to  a  minimax  approximation  problem. 

We  also  conjectured  that  this  would  hold  for  any  number  of  points,  and 

p 

this  conjecture  was  recently  confirmed  in  the  L  ,  1  <  p  £  setting  in 
S[9].  The  multidimensional  results  hold  analogously  as  shown  in  S[8]  and 
S[9]. 

Since  best  local  approximation  relates  to  Pade  apprixmants,  we  were 

lead  to  study  problems  in  the  later  area.  In  1977  [3],  we  Introduced  this 

idea  to  digital  filter  designs.  Unfortunately,  there  was  no  guarantee  on 

stability.  A  least-squares  Inverse  studied  in  1980  [3]  now  guarantees 

stability,  or  a  two-sided  approximation  scheme  (see  1977  [3]  and  S[4]) 

followed  by  certain  all-pass  filter  also  gives  stability.  To  approximate 

the  phase  we  introduced  the  e-Herglotz  tranform  which,  together  with 

methods  from  Approximation  Theory,  give  very  good  results  (cf.  S [7 ] ) . 

The  design  descussed  above  was  studied  in  the  frequency  domain.  In 

real-time  tracking,  it  is  more  convenient  to  work  in  the  time  domain. 

Following  the  suggestions  of  Mr.  W.  L.  Shepherd  and  Mr.  R.  G.  Green,  we 

studied  the  a-8-y  filter  using  limits  of  the  Kalman  gains  and 

characterized  limiting  (or  near)  optimality.  White  noise  processes  were 

considered  in  S[6]  and  recently,  in  1984  [11],  we  generalized  the  reults 

in  S [ 6 ]  to  the  color  input  setting.  This  avoids  the  singularity  as 

possibly  encounted  in  the  (colored  noise)  Kalman  filtering. 

Since  ideal  digital  filter  characteristics  are  piecewise  linear 

functions,  and  in  most  applications,  consist  only  of  the  pass  and  stop 

00 

bands,  these  characteristics  can  easily  be  smoothed  to  C  functions. 

The  approximation  order  of  such  functions  by  rational  functions  (l.e. 
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filter  realization)  with  all  poles  lying  outside  the  unit  circle  (for 
stability)  have  now  been  obtained  in  1984  [9].  In  addition,  the  order  of 
approximation  by  least-squares  inverses  which  give  an  all-pole  filter  with 
guaranteed  stability  is  also  given  in  1984  [9]. 

In  another  direction,  the  order  of  polynomial  (FIR  filters) 
approximations  on  two  disjoint  intervals  (i.e.  two  pass  bands)  was  studied 
in  1983  [5].  This  study,  however,  was  restricted  to  the  real  case  only. 

3.  Applications  of  functional  analysis  and  operator  theory. 

The  topics  in  this  section  include  approximation  by  minimum  norm 
interpolants  in  the  disc  algebra,  factorizations  of  operators,  constrained 
approximation  in  and  certain  properties  of  Toeplltz  operators.  In 

1979  [2],  we  considered  the  problem  of  minimum  norm  interolation  in  the 
disc  algebra  and  obtained  certain  rates  of  convergence.  The  main  result 
there  can  be  stated  as  follows.  Let  f  s  A  (the  disc  algebra)  and  let 
E„,  N  <*  1,2,...  be  closed  subsets  of  the  unit  circle  of  measure  zero  such 
that  ■  log(9fl_/lfl_  )  ♦  0.  Then  there  exist  minimal  norm 

N  T  h 

interpolants  sN  of  f  satisfying  (f  -  sN)  e  =  {g  e  A:  g(EN)  -  0} 
such  that  lf-8Nl  “  0(yN).  Moreover*  thi-8  rate  °f  convergence  is  sharp 
in  the  sense  that  0(yn)  cannot  be  replaced  by  o(yn). 

Papers  1982  [1],  1983  [10]  and  1984  [5)  deal  with  LU  factorizations 
of  invertible  operators  on  certain  Banach  spaces.  In  1983  [10],  a 
cholesky  factorization  for  positive  definite  bi-lnfinlte  matrices  was 
presented.  Applications  to  block  Toeplltz  matrices,  signal  processing  and 


spline  Interpolation  are  then  derived.  As  a  sample  application,  consider 

the  following  problem  in  spline  interpolation.  Let  t  be  a  bi-infinite 

partition  of  the  real  line  which  is  periodic  of  order  r.  Consider  the 

2k 

problem:  find  s  e  S.  A  L_  so  that  s(t, )  ■  x. ,  -*  <  i  <  •  with 
Ixl"  <  °°.  We  showed  that  the  subspace  of  splines  vanishing  at  all  knots 
has  a  basis  of  exponentially  growing  (or  decaying)  splines  and  hence 
derive  de  Boor’s  result  that  the  above  interpolation  has  a  unique 
soltulon.  In  another  direction  it  was  shown  that  column  diagonally 
dominant  inveritble  matrices  A  admit  LU  factorizations  and  that  L 
and  U  are  (strong)  limits  of  the  factorizations  of  the  compressions  of 
A.  It  was  also  shown  that  Gauss  elimination  performed  on  a  matrix 
"enhances"  its  diagonally  dominance.  Finally  in  1984  [3],  we  solved  a 
class  of  constrained  optimization  problems  which  lead  to  algorithms  for 
the  construction  of  convex  lnterpolants  to  convex  data. 

In  1981  [2]  and  [3],  a  factorization  theorem  for  strlcly  m-banded 
totally  positive  matrices  was  derived.  It  was  then  shown  that  such  a 
matrix  is  a  product  of  m  one-banded  matrices  with  positive  entries. 

Also  for  a  certain  class  of  block  Toeplltz  matrices,  the  smallest  sector 
containing  the  zeros  of  the  determinant  for  the  corresponding  symbol  is 
identified.  In  1984  [2]  the  null  space  of  banded  bl-infinite  block 
Toeplltz  matrices  is  determined  in  terms  of  the  Kronecher  canonical  form. 
In  1984  (4],  decay  rates  for  inverses  of  band  matrices  are  obtained.  The 
rates  are  faster  than  exponential  decay.  We  also  remark  that  in  1983  [11] 
in  S[3]  a  compendium  of  recent  developments  of  infinite  dimensional 
numerical  linear  algebra  is  given,  most  of  the  matrix  problems  dealt  with 
arose  from  the  study  of  spline  Interpolation  on  biinfinlte  meshes. 


4.  General  Approximation  Theory 

In  order  to  study  various  aspects  of  nonlinear  regression  (with 
emphasis  on  spline  methods),  we  had  to  develop  new  directions  and  obtain 
related  results  in  other  areas  of  Approximation  Theory. 

Perhaps  the  most  significant  result  here  is  the  so-called  “Problem  of 
G.  G.  Lorentz."  We  solved  this  problem  in  1978  [2],  and  even  generalized 
this  result  in  1978  [6]  and  1980  [5]  in  S[2].  The  idea  was  based  on  the 
belief  that  “like  best  approximates  like"  and  tells  us  which  subspace 
should  possibly  be  used  for  the  best  result  in  approximation.  It  has 
application  to  antenna  design,  by  indicating  the  best  location  to  place 
antennas  for  better  performance.  We  also  applied  this  idea  in  1980  [1]  to 
approximate  derivatives  from  function  values  and  proved  its  equivalence  to 
approximation  of  one  B-spllne  by  linear  combinations  of  others. 

Another  area  is  inverse  approximation.  The  basic  results  were 
obtained  in  1980  [3]  and  1982  [5].  Various  research  problems  were  posed 
In  1980  [6]  in  S(2],  One  application  Is  to  give  an  efficient  design  of  a 
stable  realizable  digital  filter,  and  another  application  is  to  give  an 
all-pass  filter  which  is  used  to  stabilize  not  necessarily  stable 
recursive  digital  filters. 

The  third  area  is  approximation  on  disjoint  intervals.  The  results 
we  obtained  1983  [5]  cannot  be  proved  by  using  one-interval  approximation 
results,  showing  that  much  research  is  needed  in  this  area.  Of  course 
there  are  applications  to  design  digital  filters  with  more  than  one  pass- 
bands  . 

As  discussed  in  Section  2,  we  also  introduced  the  idea  of  best  local 
approximation  in  trying  to  understand  local  behaviors  of  splines  with  many 


knots 
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Development  of  multivariate  splines 

Perhaps  the  most  exciting  recent  development  in  Approximation  Theory 
is  Multivariate  splines.  This  view  is  commonly  shared  by  many  leaders  in 
the  field,  including  Professor  G.  G.  Lorentz  (see  S[ 12 ]  p.  fill  of  the 
Introduction:  Approximation  and  Interpolation  in  the  last  20  years). 
(Univariate)  splines  have  proved  to  be  extremely  useful  in  the  last  two 
and  a  half  decades  in  all  applications  in  Engineering  and  Physical  and 
Biological  Sciences  that  require  data  analysis.  For  problems  in  higher 
dimensions,  tensor-product  splines  (i.e.  splines  as  linear  combination  of 
tensor-product  B-spllnes)  have  been  used.  However,  as  all  users  would 
soon  find  out  that  such  applications  are  not  only  restrictive  (since 
rectangular  grids  are  required),  their  utility  is  also  limited  due  to  the 
large  number  of  parameters  (since  coordinate  degrees  are  too  high,  for 
instance).  So,  multivariate  splines  which  are  non-tensor  product  splines 
are  required.  We  have  now  many  contributions  in  this  area  (for  instance, 
see  1982  16],  1983  [1],  [2],  [3],  [4],  16],  17],  [8],  [9]  and  1984  [1], 
(6),  [7],  [8],  and  [12]),  and  most  our  results  have  been  reported  in  the 
previous  Semi-annual  Progress  Reports.  In  this  final  report,  we  would 
like  to  report  only  those  results  not  reported  earlier. 

In  the  paper  1984  [12]  that  will  appear  in  the  Proceedings  of  the 
Second  Army  Conference  on  Applied  Mathematics  which  took  place  in  May, 
1984,  we  have  shown  that  bivariate  C*  cubic  B-splines  have  very  unusual 
behaviors,  and  in  general  require  very  large  supports.  So  to  study 
scattered  data,  it  is  advisable  to  use  higher  degree  bivariate  C* 


splines.  We  have  recently  shown  that  the  lowest  degree  for  the  existence 
of  splines  with  one  interior  grid-point  support  for  an  aribitrary 
triangulation  is  four,  and  that  degree  four  is,  however,  not  useful  for 
Lagrange-typed  interpolation.  Hence,  we  looked  into  bivariate  C1 
splines  with  total  degree  five  and  have  constructed  all  locally  supported 
splines  on  an  arbitrary  triangulation.  Using  these  splines, we  can  choose 
arbitrary  subspaces  for  interpolation  and  approximation  purposes.  For 
Instance,  if  only  the  function  and  first  partial  derivative  values  of  the 
actual  function  in  Fig.  1  are  given  at  the  sample  points  in  Fig.  2,  we 
have  constructed  the  approximating  surface  shown  in  Fig. 3  by  using  three 
locally  supported  splines  around  each  sample  point  in  the  Hermlte 
interpolation  process.  The  order  of  approximation  can  be  proved  to  be  at 
least  two.  We  are  still  improving  our  techniques  in  this  direction.  We 
mention,  in  passing  that  there  are  important  applications  to  image 
processing,  data  reduction,  image  enhancement,  etc. 
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